fix(authority): paginate canonical snapshots without mixing revisions - #4922
Conversation
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
a07903f to
a5acb89
Compare
… check Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
huangruiteng
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
Exact head: 373ae0f
English verdict: APPROVE — no blocking finding on this head.
动机:已验证旧的 one-shot canonical Todo/lease RPC 在完整集合超过 2 MiB 时失效;这会让已提升的长期 Goal 无法读取 Todo。此次交付是有用且独立可回滚的有界读传输增量,不宣称解决总体内存/IO 或后续 D1–D3。
改动思路:TypeScript 继续拥有 canonical Todo 选择和排序,复用同一 collection owner;新 page RPC 把 store identity、revision、cursor、query digest、数量与偏移绑定到 continuation,并限制每页字节和条数。Python 只校验、组装完整结果;任一页变更/异常就丢弃此前页面,不混合 revision、不回退 legacy。原 one-shot RPC 保留给既有直接调用者。
具体改动:检查了完整 base→head 28 文件差异、schema/handler/provider、Python promoted 与 pre-fence 分支、文档/RFC和相关调用者。源工作树验证:Python canonical snapshot 34 passed;TypeScript File/SQLite page 26 passed;旧路径/写入栅栏 72 passed;独立临时 PostgreSQL 16 数据库的完整 authority-store suite 288/288 passed(无 skip);control-plane typecheck 与 diff check 均通过。首次 PostgreSQL 运行的 4 个失败来自子进程误用系统旧 Python,改为工作树 uv 解释器后全套通过;不归因于此 PR。私有路径/凭据扫描无命中。
对主干的风险:最强反例是两页之间发生同内容或不同内容的 provider commit,导致表面合法的页面被拼成混合快照;负例覆盖了 revision/identity/offset 拒绝以及 Python 不返回部分结果。每页仍重读完整 head,最终 Python 结果仍占用总量内存,繁忙写入时可能反复重启;文档明确承认,这是后续按实测决定是否引入 pinned revision/streaming 的范围。没有对活动 Goal 做 promotion、迁移或原始数据改写,PostgreSQL 验证只用隔离的合成数据库。远端 CI 按本轮 review 契约不作为等待条件。
我的整体评价:未发现阻塞项。范围与已复现的 2 MiB 故障相称,规则仍归属于 TS 的 canonical owner,off/default legacy 分支有回归覆盖;可由 maintainer 合并。相关小型未来重构已通过共享 canonicalTodoCollection 完成,暂不扩展为无依据的流式框架。
huangruiteng
left a comment
There was a problem hiding this comment.
动机
修复长生命周期 Goal 的完整 Todo/依赖/租约/验收守卫读取超过 2 MiB RPC 响应上限后,健康 authority store 却无法完成公共 list/status 的问题。本条是对同一 head 上先前 review 的格式修正;先前评论缺少能力要求的独立双语段落,不应作为有效 exact-head 结论。
改动思路
由 TypeScript authority 统一决定分页和成员资格,Python 只校验并组装传输;每页绑定 store identity、revision、cursor 和 query,读期间 revision 变化则拒绝整次读取,避免混合快照。
具体改动
在 373ae0fa4add247abd316f9e4be25de21527156b 上复核了分页读取、失败形态与前次 exact-head 验证记录。本轮重跑 34 项 Python 聚焦测试及 26 项 TypeScript 分页测试,均通过;此前同一 head 的独立 PostgreSQL 集成记录为 288/288,通过 source/packaged CLI、类型、文档与边界检查。没有修改此 PR 的代码。
对主干的风险
页间仍需重读 authority head,Python 最终仍持有完整集合;因此它解决 RPC 单响应上限,不解决总 I/O、内存或繁忙写入下的重试成本。并发 revision 变化会显式失败并要求从头读,这比悄然返回混合数据安全。未把本 PR 当作 SQLite 耐久性、完整迁移或 PostgreSQL 部署资格的完成证明。本轮未查询 CI;合并仍须 maintainer 按当前 head 判断。
我的整体评价
结论:认可当前 head 的这一有界传输修复,建议 maintainer 在所需检查与独立审阅满足后合并。相关的未来向重构已体现在单一 TypeScript 分页/成员资格 owner;暂不为未证实的 I/O 优化扩展本 PR。此评论取代此前格式无效的 review,且因作者身份只能作为 COMMENTED 自审结论,不能充当 GitHub formal approval。
English verdict: APPROVE - The exact head preserves a single authority-owned collection contract and rejects mixed-revision reads; focused revalidation passed. This is a review conclusion, not merge authorization.
huangruiteng
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
动机
修复长生命周期 Goal 的完整 Todo/依赖/租约/验收守卫读取超过 2 MiB RPC 响应上限后,健康 authority store 却无法完成公共 list/status 的问题。本条是对同一 head 上先前 review 的格式修正;先前评论缺少能力要求的独立双语段落,不应作为有效 exact-head 结论。
改动思路
由 TypeScript authority 统一决定分页和成员资格,Python 只校验并组装传输;每页绑定 store identity、revision、cursor 和 query,读期间 revision 变化则拒绝整次读取,避免混合快照。
具体改动
在 373ae0fa4add247abd316f9e4be25de21527156b 上复核了分页读取、失败形态与前次 exact-head 验证记录。本轮重跑 34 项 Python 聚焦测试及 26 项 TypeScript 分页测试,均通过;此前同一 head 的独立 PostgreSQL 集成记录为 288/288,通过 source/packaged CLI、类型、文档与边界检查。没有修改此 PR 的代码。
对主干的风险
页间仍需重读 authority head,Python 最终仍持有完整集合;因此它解决 RPC 单响应上限,不解决总 I/O、内存或繁忙写入下的重试成本。并发 revision 变化会显式失败并要求从头读,这比悄然返回混合数据安全。未把本 PR 当作 SQLite 耐久性、完整迁移或 PostgreSQL 部署资格的完成证明。本轮未查询 CI;合并仍须 maintainer 按当前 head 判断。
我的整体评价
结论:认可当前 head 的这一有界传输修复,建议 maintainer 在所需检查与独立审阅满足后合并。相关的未来向重构已体现在单一 TypeScript 分页/成员资格 owner;暂不为未证实的 I/O 优化扩展本 PR。此评论取代此前格式无效的 review,且因作者身份只能作为 COMMENTED 自审结论,不能充当 GitHub formal approval。
English verdict: APPROVE - The exact head preserves a single authority-owned collection contract and rejects mixed-revision reads; focused revalidation passed. This is a review conclusion, not merge authorization.
Keep both reads added at the same seam: this branch's shared canonicalTodoCollection admission and main's exact operation-receipt readback. Take main's already-repaired bilingual mirror declaration for automatic-execution-admission-v0 so that governance repair is no longer duplicated. Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
huangruiteng
left a comment
There was a problem hiding this comment.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
动机
长生命周期 Goal 的完整 canonical 读取(全部 Todo、归档依赖、租约与验收守卫)会超过未改动的 2 MiB Python/TypeScript RPC 响应上限,使公共 list/status 在 authority store 健康的情况下直接失败。本次交付是有用且可独立回滚的有界读传输增量:为 TS T3/shared-authority L5 的读传输切片,不改变默认 provider、不晋升任何活动 Goal,也不宣称解决总体 I/O 或内存开销。
本次是本 head aba1bd7a0bf102acf98de0152b69b51d0b4cf8b2 的重新评审:相对上一轮已评审的 373ae0fa4,本 head 只加入与最新 main 的合并(保留双方在 local_authority_read.ts 同一位置新增的不同读取函数,并采用 main 已修好的 automatic-execution-admission-v0 中英镜像声明,因此本分支不再重复该治理修复)。PR 自身的产品改动未变。
改动思路
TypeScript authority 继续拥有集合准入、分页选择与验收守卫成员资格:one-shot 端点与新的 page 端点共用同一个 canonicalTodoCollection,分页只改变传输。continuation 把 Goal、store identity、provider revision、cursor、query digest 与 population(todo_count/lease_count)绑定在一起,并要求严格向前推进(lease_offset > 0 时必须已读满 todo),任一页身份、排序或 revision 不符即整体拒绝,不返回部分集合、也不回退 legacy 显示。Python 只做校验与组装,读取使用 existingOnly 打开 authority,因此 fenced 但缺失的 store 仍是既有 missing 恢复语义,读不会创建替代存储。
具体改动
26 文件 +1374/−115。Python:local_authority.py 的 promoted 读取改走新的 read_canonical_snapshot;新增 canonical_snapshot.py 组装器(逐页校验、整体失败)。TypeScript:新增 canonical_snapshot_page.ts(每页 1792 KiB/4096 条上限、continuation 编解码);local_authority_read.ts 抽出共享准入并保留原 one-shot 路径;provider 增加 existingOnly 选项;handler 注册 coordination.local_authority.todo_snapshot_page。契约:coordination_state_contract_v0.json 与双边生成绑定、生成脚本、pyproject.toml 的 mypy 列表同步。测试:Python 分页/集成/契约与调用者用例、TS 分页与 provider conformance(File、SQLite、NoKV、真实 PostgreSQL 共用同一 conformance 契约)。文档:新增 canonical snapshot pagination 参考并更新中英 RFC 与执行台账。
关键代码讲解
canonicalTodoCollection(loopx/control_plane/coordination/local_authority_read.ts):one-shot 与分页共用的唯一准入与成员资格所有者,分页不引入第二套 Todo/读模型语义。readCanonicalSnapshotPage与decodePosition(loopx/control_plane/coordination/canonical_snapshot_page.ts):只接受精确字段的 continuation,校验todo_offset/lease_offset的推进与顺序约束,并让每页落在字节与条数预算内。read_canonical_snapshot(loopx/control_plane/coordination/canonical_snapshot.py):逐页拉取并校验后组装完整结果;任一页失败即整体失败,公共结果 schema 在失败时仍然存在。- provider
existingOnly(loopx/control_plane/coordination/local_authority_provider.ts):把"读不得创建 store"从调用约定变成类型内的开关。 - 本 head 的合并解决:
canonicalTodoCollection与 main 的readLocalCoordinationOperationReceipt在同一 import 后并存,两条路由分别保留在 handler 表中。
对主干的风险
最强反例是两页之间发生 provider commit,使表面合法的页面被拼成混合快照;负例覆盖 revision、store identity、offset 与 population 不符时的整体拒绝,以及 Python 不返回部分结果。已知限制(文档已声明):每页仍重读完整 head,Python 最终仍持有完整集合,因此本 PR 限定的是 RPC 单响应上限,而不是总 provider I/O、客户端内存或繁忙写入下的重启成本;并发 revision 变化会显式失败并要求从头读取,这比静默返回混合数据安全。
验证矩阵(本轮在 aba1bd7a0 上执行):Python 契约生成检查、canonical snapshot、集成与 project-registry census 共 61 用例通过;promoted 调用方、todo 投影并发与 legacy writer fence 共 63 用例通过;TS 分页与 conformance 定向 27 用例通过;TS 全量 npm run test:control-plane 2931/2956 通过,唯一失败 tests/control_plane_ts/sqlite_capacity.test.ts 的 rehearsal 入口在未包含本 PR 的 origin/main 上以同样方式失败,属于既有环境问题;typecheck:control-plane、配置范围内 Ruff、python -m mypy(23 模块)与 examples/docs-governance-smoke.py 通过。按本 Goal 的 wait_for_ci=false 与维护者本轮要求,未抓取或等待远端 CI;合并前仍对未变 head 运行 merge-readiness。未把本 PR 当作 SQLite D2 耐久性/soak、整 Goal 迁移或 PostgreSQL 部署资格的完成证明。
语义与 CI 对齐
新增的 request/result schema 是既有 coordination 契约的扩展,通过仓库生成器产出 Python/TypeScript 双边绑定,并纳入既有契约一致性检查;持久 schema、公共 CLI 参数与 provider 默认行为未变。
我的整体评价
APPROVE:范围与已复现的 2 MiB 故障相称,规则仍归属既有 TS canonical owner,off/legacy 分支与失败形态都有回归覆盖;本 head 相对上一轮评审只增加 main 合并,未新增产品改动。相关的未来向重构已体现在单一分页/成员资格 owner,暂不为未证实的 I/O 优化扩展本 PR。剩余本地 authority 默认交付边界、SQLite D2 与 PostgreSQL 部署资格仍按现有 RFC 条件路径推进;本批准不等于 provider 晋升或控制面自合并许可。
English verdict: APPROVE - exact head aba1bd7a0bf102acf98de0152b69b51d0b4cf8b2 keeps one authority-owned collection admission behind both the one-shot and byte-bounded paged reads, rejects mixed-revision or out-of-order continuations, and merges latest main without changing the reviewed product delta; 124 focused Python cases, 27 targeted TypeScript cases, the full TypeScript suite minus one failure that reproduces on unmodified main, typecheck, Ruff, mypy and the docs governance smoke passed locally.
|
Merged decision record (admin-bypass merge, maintainer-authorized).
Change relative to the previously reviewed head Validation at the merged head:
|
…readmodel-0924 Two conflicts. The effect-runtime handler registry conflicted on imports only, so both sides keep their handlers. The shared-authority RFC conflicted on the delivery-count checkpoint: main carries the 2026-09-23 seven-boundary decomposition while this branch recorded its own dependency reconciliation. Resolution keeps main's decomposition, updates the reconciliation with the now-merged #4922/#4960/#4961 and the still-in-review #4931, and states that the estimate is updated after the combined head is accepted instead of restoring the superseded five-to-eight package range. Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Problem and result
A valid long-lived Goal could exceed the unchanged 2 MiB Python/TypeScript RPC response ceiling when a canonical read returned every Todo, archived dependency, lease and acceptance guard at once. The public list/status call then failed despite a healthy authority store. This PR assembles complete, byte-bounded pages tied to one provider incarnation, revision, cursor and query. An overlapping commit rejects the whole read; no partial collection or legacy display fallback escapes.
This is the TS T3 / shared-authority L5 read-transport slice for #4574. It does not change the default provider or promote an active Goal.
Implementation and semantics
missingwith the existing recovery response; reading never creates a replacement store. An existing head with an unreadable identity remains an identity failure. The public result schema also stays present on failure.mainintroduced a bilingual RFC without the repository-required semantic-mirror declaration. A separate two-line docs commit repairs that pre-existing governance-smoke failure; it does not change this RFC's normative contract.The bilingual RFC checkpoint lists seven principal PR boundaries including this one; after merge, six remain, or up to eight when SQLite qualification and migration each need a second reviewable batch. In-flight #4920 covers shadow-drain planning only; merged #4910 adds SQLite capacity axes but does not close the D2 soak/recovery gate. PostgreSQL service, tenant, restore and capacity qualification remains a separate medium-term lane.
Validation on
373ae0fa4add247abd316f9e4be25de21527156b33d86fc50fc36bf261c802933eaa4c40cbdb6b872dedd3c49ee5e6fdd2332e68; receiptcqr_33d86fc50fc36bf261c8verified valid. No blockers or warnings; one efficiency advisory.Each page currently reloads the full head, and Python ultimately holds the complete collection. The RPC is bounded; total provider I/O and client memory are not. Busy writers may force an explicit full-read restart. This does not qualify SQLite D2 durability/soak, persistent projection freshness, whole-Goal migration or PostgreSQL deployment.
The CLI is the affected entrypoint. Dashboard, Chat and Lark continue to consume the same assembled public list/status contracts; there is no new setting or frontend asset. This control-plane PR stays open for maintainer review and merge.